Skip to content

Revert "fix: make --continue_path work again"#133

Merged
erogol merged 1 commit into
mainfrom
revert-131-fix-continue
Dec 7, 2023
Merged

Revert "fix: make --continue_path work again"#133
erogol merged 1 commit into
mainfrom
revert-131-fix-continue

Conversation

@erogol

@erogol erogol commented Dec 7, 2023

Copy link
Copy Markdown
Member

Reverts #131

@erogol

erogol commented Dec 7, 2023

Copy link
Copy Markdown
Member Author

@eginhard I needed to revert this PR since it cause CI issues on TTS

--> EVAL PERFORMANCE
   | > Epoch:0  AvgLoss: 5.16343 
Traceback (most recent call last):
  File "/home/runner/work/TTS/TTS/TTS/bin/train_encoder.py", line 322, in <module>
    main(args)
  File "/home/runner/work/TTS/TTS/TTS/bin/train_encoder.py", line 315, in main
    _, global_step = train(model, optimizer, scheduler, criterion, train_data_loader, eval_data_loader, global_step)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/TTS/TTS/TTS/bin/train_encoder.py", line 250, in train
    best_loss = save_best_model(
                ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/trainer/io.py", line 1[83](https://github.com/coqui-ai/TTS/actions/runs/7113455600/job/19365444297?pr=3373#step:10:84), in save_best_model
    use_eval_loss = current_loss["eval_loss"] is not None and best_loss["eval_loss"] is not None

@erogol erogol merged commit 2d86257 into main Dec 7, 2023
@erogol erogol deleted the revert-131-fix-continue branch December 7, 2023 13:09
@eginhard

eginhard commented Dec 7, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

@erogol Ok... IMHO it's not really a conflict, but just shows that TTS needs to be updated for the latest Trainer version. But if you want to do it very cleanly we could do the following:

  1. TTS PR that pins Trainer==0.0.34
  2. Resubmit fix: make --continue_path work again #131 and release Trainer 0.0.35
  3. Update TTS so that tests pass and pin Trainer>=0.0.35

This was referenced Dec 7, 2023
Closed
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@erogol @eginhard